Are Loss Functions All the Same?
نویسندگان
چکیده
In this letter, we investigate the impact of choosing different loss functions from the viewpoint of statistical learning theory. We introduce a convexity assumption, which is met by all loss functions commonly used in the literature, and study how the bound on the estimation error changes with the loss. We also derive a general result on the minimizer of the expected risk for a convex loss function in the case of classification. The main outcome of our analysis is that for classification, the hinge loss appears to be the loss of choice. Other things being equal, the hinge loss leads to a convergence rate practically indistinguishable from the logistic loss rate and much better than the square loss rate. Furthermore, if the hypothesis space is sufficiently rich, the bounds obtained for the hinge loss are not loosened by the thresholding stage.
منابع مشابه
A simple approach to multiple attribute decision making using loss functions
Multiple attribute decision making (MADM) methods are very much essential in all fields of engineering, management and other areas where limited alternatives exist and the decision maker has to select the best alternative. Different methods are available in the literature to tackle the MADM problems. The MADM problems are classified as scoring methods, compromising methods and concordance metho...
متن کاملTIME-VARYING FUZZY SETS BASED ON A GAUSSIAN MEMBERSHIP FUNCTIONS FOR DEVELOPING FUZZY CONTROLLER
The paper presents a novel type of fuzzy sets, called time-Varying Fuzzy Sets (VFS). These fuzzy sets are based on the Gaussian membership functions, they are depended on the error and they are characterized by the displacement of the kernels to both right and left side of the universe of discourse, the two extremes kernels of the universe are fixed for all time. In this work we focus only on t...
متن کاملBayesian Estimation of Shift Point in Shape Parameter of Inverse Gaussian Distribution Under Different Loss Functions
In this paper, a Bayesian approach is proposed for shift point detection in an inverse Gaussian distribution. In this study, the mean parameter of inverse Gaussian distribution is assumed to be constant and shift points in shape parameter is considered. First the posterior distribution of shape parameter is obtained. Then the Bayes estimators are derived under a class of priors and using variou...
متن کاملA Fire Ignition Model and Its Application for Estimating Loss due to Damage of the Urban Gas Network in an Earthquake
Damage of the urban gas network due to an earthquake can cause much loss including fire-induced loss to infrastructure and loss due to interruption of gas service and repairing or replacing of network elements. In this paper, a new fire ignition model is proposed and applied to a conventional semi-probabilistic model for estimating various losses due to damage of an urban gas network in an eart...
متن کاملA New Approach to Solve Fully Fuzzy Linear Programming with Trapezoidal Numbers Using Conversion Functions
Recently, fuzzy linear programming problems have been considered by many. In the literature of fuzzy linear programming several models are offered and therefore some various methods have been suggested to solve these problems. One of the most important of these problems that recently has been considered; are Fully Fuzzy Linear Programming (FFLP), which all coefficients and variables of the prob...
متن کاملContinuous Discrete Variable Optimization of Structures Using Approximation Methods
Optimum design of structures is achieved while the design variables are continuous and discrete. To reduce the computational work involved in the optimization process, all the functions that are expensive to evaluate, are approximated. To approximate these functions, a semi quadratic function is employed. Only the diagonal terms of the Hessian matrix are used and these elements are estimated fr...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- Neural computation
دوره 16 5 شماره
صفحات -
تاریخ انتشار 2004